1. Field of the Invention
The present invention relates to noise reduction filters eliminating a noise signal included in an image signal, and more particularly, to a spatio-temporal and spatial joint noise reduction filters that combines more than one filters having various filter characteristics.
2. Background of the Related Art
In general, video signals initially received from a video camera and transmitted through a channel unavoidably contain noise signals. A noise signal added to a video signal provides a degraded image to viewers and has negative effects on the image signal processes. Therefore, signal receivers or any other necessary devices have a noise reduction process to avoid such problems. Some of the objectives need to be taken into consideration are as follows.
(1) Noise Elimination:
A noise signal should be eliminated so that a clear image can be provided. Particularly, an aggressive noise reduction control is necessary especially for a flat region of an image.
(2) Preservation of Edge/Detail Region:
Unfortunately, some of the edge or detail region of an image may be removed when an every region of the image goes through the aggressive noise reduction process. Therefore, a reasonable degree of noise reduction needs be carefully determined in order to preserve the edge or detail region of the image.
(3) Temporal Flickers:
A noise signal sometimes causes temporal flickers to occur between adjacent frames. This also degrades the image quality.
The first and second objectives have a relationship that one can be achieved at the expense of the other. In other words, an aggressive noise reduction performed to achieve the first objective will result in losing some of the edge/detail region of an image. On the other hand, an insufficient degree of noise reduction for achieving the second object will not satisfactorily remove the noise signal for the first objective.
In the signal processing literature, several filters have been reported for image noise suppression and image detail preservation. They are A-LMMSE (Adaptive Linear Minimum Mean Squared Error), AWA (Adaptive Weighted Average), and A-MEAN filters. The related articles are listed below:
D. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, “Adaptive Noise Smoothing Filter For Images With Signal-Dependent Noise”, IEEE Transactions on Pattern Analysis and Machine Intelligence, March 1985, vol. PAM1–7, No. 2, pp. 165–177.
M. K. Ozkan, M. I. Sezan, and A. M. Tekalp, “Adaptive Motion-Compensated Filtering of Noisy Image Sequences”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, No. 4, August 1993 at 277–290.
Carlos Polamaza-Raez and Clare D. McGillem, “Digital Phase-Locked Loop Behavior with Clock and Sampler Quantization”, IEEE Trans. on Commun., Vol. COM-33, No. 8, August 1985, pages 753 to 759.
According to the above mentioned references, a noise-contaminated input pixel signal g(i,j,t), whose horizontal, vertical, and time locations are (i,j,t), can be expressed asg(i,j,t)=f(i,j,t)+n(i,j,t)  Equation (1)where f(i,j,t) and n(i,j,t) represent an original signal and a noise signal, respectively. Therefore, a performance of a noise reduction filter depends on how accurately f(i,j,t) can be extracted from g(i,j,t). Particularly, a noise signal included in the flat region of an image is easily visible by viewers, but the noise included in the detail/edge region is not. This is called a masking effect. A-LMMSE, AWA, and A-MEAN filters may vary the level of noise reduction based on the characteristics of the noise-contaminated signal.
The A-LMMSE filter, which is originated from a linear predictor, obtains its estimated original signal {circumflex over (f)}(i,j,t) by using                                                         f              ^                        ⁡                          (                              i                ,                j                ,                t                            )                                =                                                    ϖ                ⁡                                  (                                      i                    ,                    j                    ,                    t                                    )                                            ⁢                              g                ⁡                                  (                                      i                    ,                    j                    ,                    t                                    )                                                      +                                          (                                  1                  -                                      ϖ                    ⁡                                          (                                              i                        ,                        j                        ,                        t                                            )                                                                      )                            ⁢                                                μ                  g                                ⁡                                  (                                      i                    ,                    j                    ,                    t                                    )                                                                    ⁢                                  ⁢                                  ⁢                              ϖ            ⁡                          (                              i                ,                j                ,                t                            )                                =                                                    σ                f                2                            ⁡                              (                                  i                  ,                  j                  ,                  t                                )                                                                                      σ                  f                  2                                ⁡                                  (                                      i                    ,                    j                    ,                    t                                    )                                            +                              σ                n                2                                                                        Equation  (2)            where σf2(i,j,t) is a original signal variance which is the local variance of the pixel signals included in a support region of the noise reduction filter, σn2 is the noise variance, μg(i,j,t) is the mean signal of pixel signals included in the support region. For a flat region of an image, σf2(i,j,t) is much smaller than σn2. Consequently, {overscore (ω)}(i,j,t) becomes close to zero, and the estimated original signal of the A-LMMSE filter {circumflex over (f)}(i,j,t) becomes close to μg(i,j,t). On the other hand, for an edge/detail region of the image having drastic changes σf2(i,j,t) is usually much larger than σn2. Therefore, {overscore (ω)}(i,j,t) becomes close to one, and consequently, {circumflex over (f)}(i,j,t) becomes close to the noise-contaminated input signal g(i,j,t).
In addition, the AWA and A-MEAN filters perform noise reduction using a weighted average, and each weighting factor {overscore (ω)}(l,m,t) is determined based on a difference between an input pixel signal g(i,j,t) and a pixel signal included in the support region g(l,m,t) as shown in Equation 3,4 and 5, where the (l,m,t) corresponds to a support region of the filter.                                                         f              ^                        ⁡                          (                              i                ,                j                ,                t                            )                                =                                    1              W                        ⁢                                          ∑                                  l                  ,                  m                  ,                                      t                    ∈                    S                                                              ⁢                                                ϖ                  ⁡                                      (                                          l                      ,                      m                      ,                      t                                        )                                                  ⁢                                  g                  ⁡                                      (                                          l                      ,                      m                      ,                      t                                        )                                                                                      ⁢                                  ⁢                  W          =                                    ∑                              l                ,                m                ,                                  t                  ∈                  S                                                      ⁢                          ϖ              ⁡                              (                                  l                  ,                  m                  ,                  t                                )                                                                        Equation  (3)            Each weighting factor {overscore (ω)}(l,m,t) of the AWA filter is given by                               ϖ          ⁡                      (                          l              ,              m              ,              t                        )                          =                  1                      1            +                          α              ⁢                              {                                  max                  ⁡                                      [                                          ɛ                      ,                                                                        (                                                                                    g                              ⁡                                                              (                                                                  l                                  ,                                  m                                  ,                                  t                                                                )                                                                                      -                                                          g                              ⁡                                                              (                                                                  i                                  ,                                  j                                  ,                                  t                                                                )                                                                                                              )                                                2                                                              ]                                                  }                                                                        Equation  (4)            where α and ε are usually set to α=1 and ε=2σn2. As the input pixel signal g(i,j,t) and a pixel signal included in the support region g(l,m,t) are close to each other, the corresponding weight {overscore (ω)}(l,m,t) increases. In contrast, as the difference between g(i,j,t) and g(l,m,t) is larger, {overscore (ω)}(l,m,t) decreases. On the other hand, {overscore (ω)}(l,m,t) of the A-MEAN filter is given by                               ϖ          ⁡                      (                          l              ,              m              ,              t                        )                          =                  {                                                                                          1                                                                                                      for                        ⁢                                                                                                  ⁢                                                                            x                                                                                              ≤                      c                                                                                                            0                                                                                                      for                        ⁢                                                                                                  ⁢                                                                            x                                                                                              >                      c                                                                                  ⁢                                                          ⁢              x                        =                                          g                ⁡                                  (                                      l                    ,                    m                    ,                    t                                    )                                            -                              g                ⁡                                  (                                      i                    ,                    j                    ,                    t                                    )                                                                                        Equation  (5)            where c represents a predetermined limiting factor.According to Equation (5), each weighting factor of the A-MEAN filter is set to one if the difference between g(i,j,t) and g(l,m,t) is less than or equal to c and becomes zero if the difference is larger than c. In general, c is set to c=3σn2. As shown in Equations (4) and (5), when the difference signal g(l,m,t)−g(i,j,t) is small, an aggressive noise reduction is performed by increasing {overscore (ω)}(l,m,t). On the other hand, when the difference is large, it performs a gradual noise reduction by using a lower {overscore (ω)}(l,m,t).
In conclusion, the AWA and A-MEAN filters having the characteristics different from the A-LMMSE vary the level of noise reduction based on the characteristics of an image. For example, an aggressive noise reduction is performed on the flat region of an image to eliminate the noise signal, and a moderate noise reduction is used for edge/detail preservation. However, they include the structural limitations due to their two dimensional characteristics and do not eliminate the temporal flicker mentioned earlier. If the support regions of those filters are expanded to three dimensions, the noise reduction performance may be deteriorated, resulting the blurred images. In order to avoid such problems, methods for noise reduction using motion information between image frames such as a motion adaptive noise reducer are introduced. The motion adaptive noise reducer has a reasonably good noise reduction performance while preserving a detail region of an image. However, it introduces other problems. First, the noise reduction performance of the motion adaptive noise reducer depends the accuracy of motion information. However, the noise included in the image makes it difficult to obtain the accurate information. Second, the result of the noise reduction performed on a region having motions is not satisfactory. Especially, the noise included in a region having slow motions needs to be eliminated.